Control device, medical observation system, control method, and computer readable recording medium

ABSTRACT

A control device includes: a hardware processor; and a memory, wherein the hardware processor is configured to: control an image sensor configured to generate an image signal by performing imaging sequentially according to predetermined frames; detect a frequency of vocal cord vibration of a subject based on a voice signal; set a pulse width and a light emission cycle for when a light source emits light, based on the frequency and a preset duty cycle; control the light source to emit the pulse light using the pulse width and the light emission cycle in one field period or one frame period of the image sensor in synchronization with the frequency; calculate, based on the light emission cycle or the frequency, a gain amount by which the image signal is to be multiplied; and multiply the image signal by the gain amount.

This application claims priority from Japanese Application No.2019-122380, filed on Jun. 28, 2019, the contents of which areincorporated by reference herein in its entirety.

BACKGROUND

The present disclosure relates to a control device that performs imageprocessing of images, a medical observation system, a control method,and a computer readable recording medium.

In an endoscope system, observation of the vocal cords of a subject suchas a person or an animal is sometimes performed by causing white lightto be emitted intermittently. As such an endoscope system that performsobservation of the vocal cords of a subject, a technology is known thatperforms strobe observation, in a stop state or a slow-motion state, ofvocal cords that are vibrating at high speed by emitting pulse-likeillumination light in synchronization with the vibrational frequency ofthe vocal cords. In strobe observation, the reading timing of the imagesensor and the illumination timing are asynchronous, and therefore, evenin cases where a period in which all the pixels may be commonly exposedis adequately secured, the illumination timing sometimes overlaps withthe reading timing, the exposure amount differs between images,variations in brightness between images may not be eliminated, andmaintaining image quality has not been possible.

Therefore, in Japanese Patent Publication No. 5948512, image quality maybe maintained, even in cases where pulse light is emitted with anytiming relative to the reading timing of an image sensor, by generatingan image based on a one-frame image signal (a pseudo image signal)obtained through exposure equivalent to one pulse of the pulse light,based on first pixel signals, which are one frame's worth of an imagesignal and read out with reading timing that includes a pulse-lightillumination period, and one frame's worth of second pixel signals,which are read out with reading timing after one frame of the firstpixel signals, the image being generated by synthesizing pixel signalswhich correspond to an overlap line among the first pixel signals andpixel signals which correspond to an overlap line among the second pixelsignals, wherein an overlap line is configured by an overlap between apulse-light illumination period and reading timing among a plurality ofpixels in an image sensor.

SUMMARY

However, in Japanese Patent Publication No. 5948512, there is a problemin that, when the duty cycle indicating the ratio between one frameperiod and the total emission period in the one frame period is fixed,the pulse-light emission time is inversely proportional to the vocalcord frequency, and hence the higher the vocal cord frequency, theshorter the pulse-light emission time is, whereby the exposure amount isthen small and the image grows dark.

There is a need for a control device that performs image processing ofimages, a medical observation system, a control method, and a computerreadable recording medium that enable image brightness to be maintainedconstant regardless of a vocal cord frequency.

According to one aspect of the present disclosure, there is provided acontrol device including: a hardware processor; and a memory, whereinthe hardware processor is configured to: control an image sensorincluding a plurality of pixels arranged in a two-dimensional matrix togenerate an image signal by performing imaging sequentially according topredetermined frames; detect a frequency of vocal cord vibration of asubject based on a voice signal input from an external device; set, fora light source configured to intermittently emit pulse light accordingto a pulse current, a pulse width and a light emission cycle for whenthe light source emits light, based on the frequency and a preset dutycycle; control the light source to emit the pulse light using the pulsewidth and the light emission cycle in one field period or one frameperiod of the image sensor in synchronization with the frequency;calculate, based on the light emission cycle or the frequency, a gainamount by which the image signal is to be multiplied; and multiply theimage signal by the gain amount.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a function configuration of anendoscope system according to a first embodiment;

FIG. 2 is a diagram illustrating an overview of a correction method ofcorrecting the brightness of a pseudo frame based on a pseudo imagesignal as performed by a control unit according to the first embodiment;

FIG. 3 is a flowchart illustrating an overview of processing executed byan endoscope system according to the first embodiment;

FIG. 4 is a block diagram illustrating a function configuration of anendoscope system according to a second embodiment;

FIG. 5 is a diagram that schematically illustrates a method ofgenerating a pseudo frame as performed by a first image processoraccording to the second embodiment; and

FIG. 6 is a diagram illustrating an overview of a correction method ofcorrecting the brightness of a pseudo frame based on a pseudo imagesignal as performed by a control unit according to the secondembodiment.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in detailhereinbelow in conjunction with the drawings. Note that the presentdisclosure is not limited to or by the following embodiments.Furthermore, the respective drawings referenced in the followingdescription merely provide an approximate illustration of shapes, sizes,and positional relationships to an extent enabling an understanding ofthe content of the present disclosure. That is, the present disclosureis not limited to or by the shapes, sizes, and positional relationshipsillustrated in the respective drawings. Furthermore, the disclosures ofthe drawings are described by assigning the same reference signs to thesame parts. In addition, as an example of a medical observation systemaccording to the present disclosure, an endoscope system capable ofobserving vocal cords will be described.

First Embodiment

Overall Configuration of Endoscope System

FIG. 1 is a block diagram illustrating a function configuration of anendoscope system according to a first embodiment. The endoscope system 1illustrated in FIG. 1 is used in the medical field and enables strobeobservation of vocal cords serving as an object. The endoscope system 1includes an endoscope 2, an input device 3, a voice input device 4, adisplay device 5, and a control device 6.

The endoscope 2 is inserted into a subject such as a living body. Theendoscope 2 projects emitted light (pulse light) supplied from thecontrol device 6, described subsequently, toward the object from the endportion of an insertable section 20, and generates an image signal foran object image by receiving light that is reflected from the object.The endoscope 2 includes an imaging unit 21 at least at the end portionof the insertable section 20.

The imaging unit 21 includes an optical system 211 and an image sensor212. The optical system 211 includes one or a plurality of lenses andforms an object image from the object on a light-receiving surface ofthe image sensor 212. Based on control by the control device 6, theimage sensor 212 receives the object image formed by the optical system211 according to a predetermined frame rate and generates an imagesignal by photoelectrically converting the received object image. Theimage sensor 212 outputs the image signal to the control device 6. Theimage sensor 212 includes a charge coupled device (CCD) sensor of aninterlace scan system. Furthermore, the image sensor 212 includes aplurality of pixels arranged in a two-dimensional matrix.

The input device 3 receives user operations by a user such as a doctorand outputs operation signals that correspond to the received useroperations to the control device 6. The input device 3 includes akeyboard, a mouse, switches, buttons, a foot switch and a touch panel,or the like.

Based on control by the control device 6, the voice input device 4receives inputs of voice emitted from the vocal cords and makes outputsto the control device 6 by converting this voice to a voice signal. Thevoice input device 4 includes a microphone or the like.

Based on control by the control device 6, the display device 5 displaysa display image based on a video signal which is input from the controldevice 6. The display device 5 includes a display that employs liquidcrystals or organic electroluminescence (EL), or the like.

The control device 6 includes at least one or more processors thatinclude a memory and hardware such as a central processing unit (CPU),an field-programmable gate array (FPGA), and a graphics processing unit(GPU). The control device 6 integrally controls the operations of eachof the parts included in the endoscope system 1. The control device 6includes a light source unit 60, a light source driving unit 61, a firstimage processor 62, a brightness detection unit 63, an autogaincontroller unit 64 (called “AGC unit 64” hereinbelow), a second imageprocessor 65, a preprocessor 66, a vocal cord frequency detection unit67, a memory 68, and a control unit 69.

The light source unit 60 emits pulse light intermittently using apredetermined intensity and light emission cycle based on a drivecurrent which is input from the light source driving unit 61. The lightsource unit 60 includes a white LED (light emitting diode) and acondenser lens, or the like. The pulse light emitted by the light sourceunit 60 is emitted toward an object from the end of the endoscope 2.Note that the light source unit 60 is not limited to a white LED and maybe configured capable of emitting white light by combining light whichis emitted by a red LED, a blue LED, and a green LED, respectively.

Based on control by the control unit 69, the light source driving unit61 causes the light source unit 60 to emit pulse light in apredetermined light emission cycle and to emit the pulse light at apredetermined intensity. Specifically, the light source driving unit 61causes the light source unit 60 to emit light in a light amountcorresponding to a predetermined pulse current value based on a PWMcontrol signal and a current value control signal which are input fromthe control unit 69.

Based on the control unit 69, the first image processor 62 performsvarious image processing on the image signal which is input from theimage sensor 212 of the endoscope 2 and makes outputs to the brightnessdetection unit 63. Here, the various image processing by the first imageprocessor 62 is, with respect to a pixel signal, A/D conversionprocessing, optical black subtraction processing, white balance (WB)adjustment processing, demosaic processing (in the case of aconfiguration in which the image sensor 212 includes a Bayer array colorfilter (not illustrated)), color matrix arithmetic processing, gammacorrection processing, color reproduction processing, and edgeenhancement processing, and the like. In addition, the first imageprocessor 62 generates a pseudo image signal based on light sourceinformation that is input from the control unit 69 and that relates tothe light emission cycle of the light source unit 60, the current valuesupplied to the light source unit 60, and the pulse width of the pulselight. Specifically, the first image processor 62 generates, from animage signal of a plurality of continuous frames, a pseudo pixel signalthat corresponds to a pixel signal in a case where all the pixels of theimage sensor 212 are exposed in a pulse-light light emission period(light emission time) by the light source unit 60. More specifically,when generating the pseudo pixel signal, the first image processor 62generates the pseudo pixel signal by using, for a specified horizontalline among horizontal lines of a plurality of pixels in the image sensor212, an image signal which is obtained by multiplying a specified oneframe's worth of an image signal from the respective pixels of thespecified horizontal line by the ratio, of a specified pulse-lightexposure amount resulting from exposure in a specified one frame, to thetotal exposure amount obtained by adding up all of the respectivepulse-light exposure amounts resulting from exposure of the specifiedhorizontal line in one specified frame that contains a specifiedpulse-light illumination period.

The brightness detection unit 63 detects the brightness of an imagebased on an image signal or a pseudo image signal (an image signal)which is input from the first image processor 62 and outputs brightnessinformation relating to this brightness to the control unit 69. Here,brightness information is an average value for the luminance of eachpixel in the image based on the pseudo image signal or image signal.Furthermore, the brightness detection unit 63 outputs the pseudo imagesignal, which has been input from the first image processor 62, to theAGC unit 64.

Based on the control unit 69, the AGC unit 64 adjusts the gain of thepseudo image signal, which has been input from the brightness detectionunit 63, and the gain of a pseudo image signal to be output to thesecond image processor 65, to predetermined values and outputs thepseudo image signal to the second image processor 65. Specifically, theAGC unit 64 multiplies the image signal or the pseudo image signal bythe gain amounts set by the control unit 69 and outputs the image signalor the pseudo image signal to the second image processor 65.

Based on the control unit 69, the second image processor 65 generates avideo signal by performing various image processing on the pseudo imagesignal that has been input from the AGC unit 64 and outputs the videosignal to the display device 5. For example, when the pseudo imagesignal is RGB data, the second image processor 65 performs imageprocessing to convert RGB data to YCU data.

The preprocessor 66 amplifies a voice signal that has been input fromthe voice input device 4, performs A/D conversion processing on theamplified voice signal, and outputs the A/D converted signal to thevocal cord frequency detection unit 67. Note that the functions of thepreprocessor 66 may also be provided in the voice input device 4.

Based on the voice signal that has been input from the preprocessor 66,the vocal cord frequency detection unit 67 detects the frequency ofvibration of voice which is input to the voice input device 4.Specifically, the vocal cord frequency detection unit 67 detects a vocalcord vibrational frequency based on the voice signal that has been inputfrom the preprocessor 66 and outputs the detected vocal cord vibrationalfrequency to the control unit 69.

The memory 68 records various information and the like which is executedby the endoscope system 1. The memory 68 includes a volatile memory, anonvolatile memory, and a removable memory card, or the like.Furthermore, the memory 68 includes a program recording unit 681 thatrecords various programs executed by the endoscope system 1.

The control unit 69 includes at least one or more processors thatinclude a memory and hardware such as a CPU. The control unit 69 causesthe image sensor 212 to generate an image signal through sequentialimaging according to a predetermined frame rate of, for example, 60 fpsor 50 fps and to output the image signal to the first image processor62. Furthermore, the control unit 69 sets, for the light source unit 60,the pulse width (light emission time) and the light emission cycle(light emission frequency) for when the light source unit 60 emitslight, based on the vocal cord vibrational frequency that has been inputfrom the vocal cord frequency detection unit 67, and a preset dutycycle. In addition, the control unit 69 causes the light source unit 60to emit pulse light toward the vocal cords using the pulse width and thelight emission cycle which have been set and within one field period orone frame period of the image sensor 212 in synchronization with thevocal cord vibrational frequency which has been input from the vocalcord frequency detection unit 67. Furthermore, the control unit 69 sets,based on the light emission cycle of the light source unit 60, the gainamount by which the pseudo image signal is to be multiplied by the AGCunit 64.

Overview of Brightness Correction

A method of correcting the brightness of a pseudo field based on apseudo image signal as performed by the control unit 69 will bedescribed next. FIG. 2 is a diagram illustrating an overview of acorrection method of correcting the brightness of a pseudo field basedon a pseudo image signal as performed by the control unit 69. In FIG. 2,starting at the top, (a) illustrates the vocal cord vibrationalfrequency, (b) illustrates the exposure period (light emission cycle(light emission frequency) and pulse width (light emission time)) in onefield of the image sensor 212, (c) illustrates the brightness of onefield based on a pseudo image signal, (d) illustrates 0 dB (gain amount)(light emission periodicity [Hz]/frame rate [Hz]) of the AGC unit 64,and (e) illustrates the brightness of an output image (the pseudo fieldbrightness×0 dB gain).

As illustrated in FIG. 2, the control unit 69 sets a pulse width and alight emission cycle for when the light source unit 60 emits pulse lightin one field period of the image sensor 212, based on the vocal cordvibrational frequency detected by the vocal cord frequency detectionunit 67 and the duty cycle, and causes the light source unit 60 to emitlight using the set pulse width and light emission cycle. Specifically,as illustrated in FIG. 2, when the vocal cord vibrational frequencydetected by the vocal cord frequency detection unit 67 is 600 Hz and theduty cycle is 50%, the control unit 69 causes the light source unit 60to emit light using the pulse width P1 in one field period (exposure A0)of the image sensor 212 by supplying a PWM control signal and a currentvalue control signal, for which a light emission cycle of 600 Hz is set,to the light source driving unit 61. In this case, the first imageprocessor 62 generates a pseudo image signal so as to establish theexposure amount of the latest pulse light emitted by the light sourceunit 60 in one field period of the image signal generated by the imagesensor 212. That is, the brightness of the pseudo field based on thepseudo image signal generated by the first image processor 62corresponds to the exposure amount of the latest pulse light when thelight source unit 60 emits one pulse (for example, pulse light of pulsewidth P1, pulse light of pulse width P2, pulse light of pulse width P3,pulse light of pulse width P4) in each single field period.

In the case illustrated in FIG. 2, the brightness of the pseudo fieldmay be represented by the following equation (1) when the duty cycle isa preset fixed value of 50%, for example.

Brightness of pseudo field=(1/(light emission cycle))×duty cycle  (1)

For example, when a pulse width P1 and a light emission cycle of 600 Hzare set in one field period (exposure A0) of the image sensor 212, thebrightness of the pseudo field (exposure A0) is as follows:

The brightness of pseudo field F1=( 1/600 Hz)×duty cycle.

Further, when a pulse width P2 and a light emission cycle of 300 Hz areset in the next one field period (exposure B0) of the image sensor 212,the brightness of the next pseudo field (exposure B0) is as follows:

Brightness of pseudo field F2=( 1/300 Hz)×duty cycle.

Thus, when the duty cycle is fixed, the light emission time is inverselyproportional to the vocal cord vibrational frequency, and therefore apseudo field-based image grows dark as the vocal cord vibrationalfrequency rises. Hence, when the vocal cord vibrational frequency is x,the control unit 69 multiplies the pseudo image signal by the gain of 0dB in the following equation (2) and, by eliminating dependence on thevocal cord vibrational frequency, keeps the image brightness constant.

0 dB gain=x Hz/frame rate (Hz) of image sensor 212  (2)

For example, in a case where, in one field period (exposure A0) of theimage sensor 212, the gain amount, by which the pseudo field F1 orpseudo frame based on the pseudo image signal generated by the imagesensor 212 is multiplied, is 0 dB and where the light emission cycle ofvocal cord vibrations is 600 Hz and the frame rate of the image sensor212 is 60 Hz, the equation is as follows.

0 db=600 Hz/60 Hz

Furthermore, the brightness of the output image based on thegain-adjusted pseudo image signal may be represented by the followingequation (3):

Output image brightness=pseudo field brightness×0 dB gain  (3)

That is, according to equations (1) and (2), the output imagebrightness=( 1/60 Hz)×duty cycle.

Thus, as illustrated in FIG. 2, the control unit 69 sets, with respectto the AGC unit 64, a gain amount that corresponds to each pseudo fieldfor the AGC unit 64 and causes the AGC unit 64 to multiply the pseudoimage signal corresponding to each pseudo field by 0 dB (the gainamount). Accordingly, the image brightness may be kept constant byeliminating dependence on the vocal cord vibrational frequency.

Endoscope System Processing

The processing executed by the endoscope system 1 will be describednext. FIG. 3 is a flowchart illustrating an overview of processingexecuted by the endoscope system 1.

As illustrated in FIG. 3, the vocal cord frequency detection unit 67first detects the vocal cord vibrational frequency based on a voicesignal that is input from the voice input device 4 via the preprocessor66 (step S101).

Next, the control unit 69 sets the pulse width and light emission cyclefor when the light source unit 60 emits pulse light based on the vocalcord vibrational frequency detected by the vocal cord frequencydetection unit 67 (step S102).

Thereafter, the control unit 69 causes the light source unit 60 to emitlight in synchronization with the vocal cord vibrational frequency andusing the pulse width and the light emission cycle which have been setin step S102 (step S103). Specifically, the control unit 69 causes thelight source unit 60 to emit light in synchronization with the vocalcord vibrational frequency and using the pulse width and the lightemission cycle set in step S102, by supplying a PWM control signal and acurrent value control signal, for which the pulse width and the lightemission cycle have been set, to the light source driving unit 61.

Next, the control unit 69 causes the image sensor 212 to form an imageas a result of receiving the reflected light from an object resultingfrom the light source unit 60 emitting pulse light (step S104). In thiscase, the image sensor 212 outputs an image signal to the first imageprocessor 62.

Thereafter, the first image processor 62 generates a pseudo image signalby using an image signal that is input from the image sensor 212 and animage signal that is input from the image sensor 212 before the formerimage signal (step S105).

Next, the control unit 69 sets a gain amount using the AGC unit 64 basedon the light emission cycle set in step S102 (step S106).

Thereafter, the AGC unit 64 amplifies the pseudo image signal which hasbeen input from the first image processor 62 via the brightnessdetection unit 63, by the gain amount set by the control unit 69 andoutputs the amplified signal to the second image processor 65 (stepS107).

Subsequently, when an end signal to end the subject inspection has beeninput from the input device 3 (step S108: Yes), the endoscope system 1ends the processing. In contrast, when an end instruction to end thesubject inspection has not been input from the input device 3 (stepS108: No), the endoscope system 1 returns to step S101 above.

According to the first embodiment, the control unit 69 sets, withrespect to the AGC unit 64, a gain amount that corresponds to eachpseudo field for the AGC unit 64 and causes the AGC unit 64 to multiplythe pseudo image signal corresponding to each pseudo field by 0 dB (thegain amount), and therefore the image brightness may be kept constantirrespective of the vocal cord frequency.

Second Embodiment

A second embodiment will be described next. In addition to having adifferent configuration from the endoscope system 1 according to thefirst embodiment, an endoscope system according to the second embodimentutilizes a different method of generating a pseudo image signal asgenerated by the first image processing unit, and different brightnesscorrection. After describing the configuration of the endoscope systemaccording to the second embodiment hereinbelow, the method of generatinga pseudo image signal as generated by the first image processing unit,and brightness correction, in the endoscope system according to thesecond embodiment will be described. Note that the same reference signsare assigned to the same configurations as in the endoscope system 1according to the foregoing first embodiment and that detaileddescriptions of these configurations are omitted.

Endoscope System Function Configuration

FIG. 4 is a block diagram illustrating a function configuration of anendoscope system according to the second embodiment. The endoscopesystem 1A illustrated in FIG. 4 includes an endoscope 2A instead of theendoscope 2 of the endoscope system 1 according to the first embodimenthereinabove. The endoscope 2A includes an image sensor 212A instead ofthe image sensor 212 of the endoscope 2 according to the firstembodiment hereinabove.

Based on control by the control device 6, the image sensor 212A receivesthe object image formed by the optical system 211 according to apredetermined frame rate and generates an image signal byphotoelectrically converting the received object image. The image sensor212A outputs the image signal to the control device 6. The image sensor212A includes a progressive CMOS (Complementary Metal OxideSemiconductor) sensor, or the like. The image sensor 212A has aplurality of pixels arranged in a two-dimensional matrix shape.

Method of Generating Pseudo Frames Using First Image Processing Unit

A method of generating pseudo frames using the first image processor 62for the image signal generated by the image sensor 212A will bedescribed next. FIG. 5 is a diagram that schematically illustrates amethod of generating pseudo frames using the first image processor 62.In FIG. 5, starting at the top, (a) illustrates exposure timing for eachframe, (b) illustrates sensor outputs from the image sensor 212A, and(c) to (e) illustrate pseudo image signals of each line. Furthermore, inFIG. 5, pulse-like regions represent pulse light PLS0 to pulse lightPLS5 as emitted by the light source unit 60. Furthermore, in FIG. 5, onerectangular block B1 corresponds to one line period in the verticaldirection.

Case where Dimming is not Performed

First, a case where dimming of the pulse light emitted by the lightsource unit 60 is not performed will be described. The first imageprocessor 62 selects generation-target pulse light from a cycle of atleast two frames (a pseudo frame generation period), which has beeninput from the image sensor 212A) and provides the pseudo framegeneration period in a cycle of at least two frames. Furthermore, thetarget pulse light is assumed to be the latest on the time axis and tobe contained within the pseudo frame generation period. Specifically, asillustrated in FIG. 5, because only a portion of the pulse light PLS4 iscontained in the pseudo frame generation period, the pulse light PLS3 isselected as the target pulse light. Hence, the first image processor 62combines at least two frames' worth of the image signal at a ratiocorresponding to the amount of exposure to pulse light, therebygenerating pseudo frames based on the pseudo image signal.

Line X Case

First, a case involving line X illustrated in FIG. 5 will be described.

As illustrated in FIG. 5, line X is exposed in a two-line period bypulse light PLS2, in a seven-line period by pulse light PLS3, and in asix-line period by pulse light PLS4. Hence, the first image processor 62calculates a [pseudo frame line X image] constituting a line-X pseudoframe by using equation (4) below, said image being regarded as an imagethat corresponds to exposure in a seven-line period by pulse light PLS3in a [RAW_G_0_IN line X image].

$\begin{matrix}{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{20mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack = {\frac{7}{2 + 7 + 6}\left\lbrack {{RAW\_ G}\_ 0{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack}} & (4)\end{matrix}$

Line Y Case

Next, a case involving a line Y as illustrated in FIG. will bedescribed.

As illustrated in FIG. 5, line Y is exposed in a two-line period bypulse light PLS1, in a seven-line period by pulse light PLS2, and in afour-line period by pulse light PLS3. In addition, line Y is exposed ina three-line period by pulse light PLS3, in a seven-line period by pulselight PLS4, and in a three-line period by pulse light PLS5. Hence, thefirst image processor 62 calculates a [pseudo frame line Y image]constituting a line-Y pseudo frame by using equation (5) below, saidimage being regarded as an image obtained by combining an image thatcorresponds to exposure in a four-line period by pulse light PLS3 in a[RAW_G_1_IN line X image] with an image that corresponds to exposure ina three-line period by pulse light PLS3 in a [RAW_G_0_IN line X image].

$\begin{matrix}{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{14mu} {LINE}\mspace{14mu} Y\mspace{14mu} {IMAGE}} \right\rbrack = {{\frac{4}{2 + 7 + 4}\left\lbrack {{RAW\_ G}\_ 1{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack} + {\frac{3}{3 + 7 + 3}\left\lbrack {{RAW\_ G}\_ 0{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack}}} & (5)\end{matrix}$

Line Z Case

Next, a case involving a line Z as illustrated in FIG. will bedescribed.

As illustrated in FIG. 5, line Z is exposed in a seven-line period bypulse light PLS2, and in a seven-line period by pulse light PLS3. Hence,the first image processor 62 calculates a [pseudo frame line Z image]constituting a line-Z pseudo frame by using equation (6) below, saidimage being regarded as an image that corresponds to exposure in aseven-line period by pulse light PLS3 in a [frame one-line Z image].

$\begin{matrix}{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{14mu} {LINE}\mspace{14mu} Z\mspace{14mu} {IMAGE}} \right\rbrack = {\frac{7}{7 + 7}\left\lbrack {{RAW\_ G}\_ 1{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack}} & (6)\end{matrix}$

Case where Dimming is Performed

Next, a case where dimming of the pulse light emitted by the lightsource unit 60 is performed will be described. The first image processor62 generates a pseudo frame through summation according to the ratiobetween the exposure amounts of the respective pulse light.Specifically, when there is no dimming of the light source unit 60, thefirst image processor 62 generates a pseudo frame by using exposureamounts as light emission periods, but when dimming of the light sourceunit 60 is performed, the first image processor 62 generates a pseudoframe by multiplying light amounts in light emission periods by theexposure amounts. Hence, the first image processor 62 calculates a[pseudo frame line X image], a [pseudo frame line Y image], and a[pseudo frame line Z image] by using equations (7) to (9) below.

$\begin{matrix}{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{20mu} {IMAGE}} \right\rbrack = {\frac{7*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}{{2*{PLS}\; 2\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}} + {7*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}}\left\lbrack \left\lbrack {{RAW\_ G}\_ 0{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack \right\rbrack}} & (7) \\{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{14mu} {LINE}\mspace{14mu} Y\mspace{14mu} {IMAGE}} \right\rbrack = {\frac{4*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}{{2*{PLS}\; 1\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}} + {7*7*{PLS}\; 2\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}} + {4*{PLS3}\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}}{\quad{\left\lbrack {{RAW\_ G}\_ 1{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack + \frac{3*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}{{3*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}} + {7*{PLS}\; 4\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}} + {3*{PLS}\; 5\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}}}}}} & (8) \\{\left\lbrack {{PSEUDO}\mspace{14mu} {FRAME}\mspace{14mu} {LINE}\mspace{14mu} Z\mspace{14mu} {IMAGE}} \right\rbrack = {\frac{7*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}{{7*{PLS}\; 2\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUN}} + {7*{PLS}\; 3\mspace{14mu} {LIGHT}\mspace{14mu} {AMOUNT}}}\left\lbrack {{RAW\_ G}\_ 1{\_ IN}\mspace{14mu} {LINE}\mspace{14mu} X\mspace{14mu} {IMAGE}} \right\rbrack}} & (9)\end{matrix}$

Thus, the first image processor 62 uses an image signal of at least twoframes to generate a pseudo frame (a pseudo image) based on a pseudoimage signal that corresponds to the pulse light of one pulse.

Overview of Brightness Correction

A method of correcting the brightness of a pseudo frame based on apseudo image signal as performed by the control unit 69 will bedescribed next. FIG. 6 is a diagram illustrating an overview of acorrection method of correcting the brightness of a pseudo frame basedon a pseudo image signal as performed by the control unit 69. In FIG. 6,starting at the top, (a) illustrates the vocal cord vibrationalfrequency, (b) illustrates the exposure period (light emission cycle(light emission frequency) and pulse width (light emission time) in oneframe of the image sensor 212), (c) illustrates the brightness of oneframe based on a pseudo image signal, (d) illustrates 0 dB (gain amount)(light emission periodicity [Hz]/frame rate [Hz]) by the AGC unit 64,and (e) illustrates the output image brightness (pseudo framebrightness×0 dB gain).

As illustrated in FIG. 6, as per the foregoing first embodiment, thecontrol unit 69 sets a pulse width and a light emission cycle for whenthe light source unit 60 emits pulse light in the frame period (exposure0) of the image sensor 212A, based on the vocal cord vibrationalfrequency detected by the vocal cord frequency detection unit 67 and theduty cycle, and causes the light source unit 60 to emit light using theset pulse width and light emission cycle. The control unit 69 sets, forthe AGC unit 64, a gain amount that corresponds to each pseudo frame forthe AGC unit 64 and causes the AGC unit 64 to multiply the pseudo imagesignal corresponding to each pseudo frame by 0 dB (the gain amount).Accordingly, the image brightness may be kept constant by eliminatingdependence on the vocal cord vibrational frequency.

According to the second embodiment, the control unit 69 sets, for theAGC unit 64, a gain amount that corresponds to each pseudo field for theAGC unit 64 and causes the AGC unit 64 to multiply the pseudo imagesignal corresponding to each pseudo field by 0 dB (the gain amount), andtherefore the image brightness may be kept constant irrespective of thevocal cord frequency.

Furthermore, according to the second embodiment, the first imageprocessor 62 uses an image signal of at least two frames to generate apseudo frame (a pseudo image) based on a pseudo image signal thatcorresponds to the pulse light of one pulse, and therefore the imagebrightness suitable for observation may be kept constant.

In addition, according to the second embodiment, when the first imageprocessor 62 generates the pseudo frame, it is possible to reliablyprevent exposure unevenness because a specified pulse-light lightemission period is, in one field period or one frame period of the imagesensor 212A, the period of the latest pulse light emitted by the lightsource unit 60 and is the period in which the pseudo image signal isgenerated.

Further Embodiments

By suitably combining a plurality of constituent elements which aredisclosed in the endoscope systems according to the foregoing first andsecond embodiments of the present disclosure, various embodiments may beconfigured. For example, several constituent elements may be removedfrom among all the constituent elements disclosed in the endoscopesystems according to the foregoing first and second embodiments of thepresent disclosure. Moreover, the constituent elements described in theendoscope systems according to the foregoing first and secondembodiments of the present disclosure may also be suitably combined.

Furthermore, “parts” mentioned earlier in the endoscope systemsaccording to the first and second embodiments of the present disclosuremay also be replaced with the wording “means” and “circuits”, and soforth. For example, “control unit” may be replaced with “control means”or “control circuit”.

In addition, a program that is executed by the endoscope systemsaccording to the first and second embodiments of the present disclosureis file data that is in an installable format or an executable formatand that is provided by being recorded on a recording medium that iscomputer-readable, such as a CD-ROM, a flexible disk (FD), a CD-R, adigital versatile disk (DVD), a USB medium, or a flash memory.

Furthermore, a program that is executed by the endoscope systemsaccording to the first and second embodiments of the present disclosuremay be configured to be stored on a computer connected to a network suchas the internet or to be provided by being downloaded over the internet.

Note that, although expressions such as “first”, “thereafter”, and“next” are used in the flowchart descriptions in the presentspecification to illustrate the pre- and post-processing relationshipsbetween steps, the sequence of the processing for carrying out thepresent disclosure is not necessarily uniquely defined by suchexpressions. In other words, the sequence of the processing in theflowcharts disclosed in the present specification may be revised withina consistent scope.

The present disclosure exhibits an advantageous effect of enabling imagebrightness to be maintained constant regardless of a vocal cordfrequency.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. A control device comprising: a hardwareprocessor; and a memory, wherein the hardware processor is configuredto: control an image sensor including a plurality of pixels arranged ina two-dimensional matrix to generate an image signal by performingimaging sequentially according to predetermined frames; detect afrequency of vocal cord vibration of a subject based on a voice signalinput from an external device; set, for a light source configured tointermittently emit pulse light according to a pulse current, a pulsewidth and a light emission cycle for when the light source emits light,based on the frequency and a preset duty cycle; control the light sourceto emit the pulse light using the pulse width and the light emissioncycle in one field period or one frame period of the image sensor insynchronization with the frequency; calculate, based on the lightemission cycle or the frequency, a gain amount by which the image signalis to be multiplied; and multiply the image signal by the gain amount.2. The control device according to claim 1, wherein the hardwareprocessor is configured to: generate a pseudo image signal thatcorresponds to the image signal when the plurality of pixels are exposedin a specified pulse-light light emission period, based on the imagesignal generated by the image sensor; and multiply the pseudo imagesignal by the gain amount.
 3. The control device according to claim 2,wherein the hardware processor is configured to: generate a pseudo imagesignal that corresponds to the image signal when the plurality of pixelsare exposed in a specified pulse-light light emission period, based ontwo or more temporally consecutive image signals generated by the imagesensor; and multiply the pseudo image signal by the gain amount.
 4. Thecontrol device according to claim 3, wherein the specified pulse-lightlight emission period is, in one field period or one frame period of theimage sensor, the period of the latest pulse light emitted by the lightsource unit and the period in which the pseudo image signal isgenerated.
 5. The control device according to claim 1, wherein the imagesensor is a CCD sensor.
 6. The control device according to claim 1,wherein the image sensor is a CMOS sensor.
 7. A medical observationsystem comprising: the control device according to claim 1; an endoscopeincluding the image sensor configured to generate the image signal, theimage sensor being disposed at an end portion of an insertable sectioninserted into a subject; and a voice input device configured to receivea voice input and generate the voice signal, wherein, by supplying pulselight to the endoscope, the light source is configured to project thepulse light toward the object from the end.
 8. A control method,comprising: controlling an image sensor having a plurality of pixelsarranged in a two-dimensional matrix shape to generate an image signalby performing imaging sequentially according to predetermined frames;detecting a frequency of vocal cord vibration of a subject based on avoice signal which is input from an external device; setting, for alight source configured to intermittently emit pulse light according toa pulse current, a pulse width and a light emission cycle for when thelight source emits light, based on the frequency and a preset dutycycle; controlling the light source to emit the pulse light using thepulse width and the light emission cycle in one field period or oneframe period of the image sensor in synchronization with the frequency;calculating, based on the light emission cycle or the frequency, thegain amount by which the image signal is to be multiplied; andmultiplying the image signal by the gain amount.
 9. A non-transitorycomputer-readable recording medium on which an executable program isrecorded, the program causing a processor of a control device toexecute: controlling an image sensor having a plurality of pixelsarranged in a two-dimensional matrix shape to generate an image signalby performing imaging sequentially according to predetermined frames;detecting a frequency of vocal cord vibration of a subject based on avoice signal which is input from an external device; setting, for alight source configured to intermittently emit pulse light according toa pulse current, a pulse width and a light emission cycle for when thelight source emits light, based on the frequency and a preset dutycycle; controlling the light source to emit the pulse light using thepulse width and the light emission cycle in one field period or oneframe period of the image sensor in synchronization with the frequency;calculating, based on the light emission cycle or the frequency, thegain amount by which the image signal is to be multiplied; andmultiplying the image signal by the gain amount.